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ABSTRACT 

Our  ultimate  goat  is  to  develop  neural-like  cognitive  sensory  processing  within  non -neuronal  system* 
Foward  this  end,  computational  models  are  being  developed  for  selectively  attending  the  task-reles ant  rs 
of  cuinpusite  sensory  excitations  in  an  example  sound  processing  application.  Significant  stimuli  partial- 
are  selectively  attended  through  the  use  of  generalized  neural  adaptive  beamformers.  Computational 
components  are  being  tested  by  experiment  in  the  laboratory  and  also  by  use  of  recordings  from  sensm 
deployments  in  the  ocean.  Results  will  be  presented.  These  computational  components  are  being 
integrated  into  a  comprehensive  processing  architecture  that  simultaneously  attends  memory  according  Pj 
stimuli,  attends  stimuli  according  to  memory,  and  attends  stimuli  and  memory  according  to  an  ongoing 
thought  process.  The  proposed  neural  architecture  is  potentially  very  fast  when  implemented  in  spcci.ii 
hardware. 


1.  rNTRODUCTlON 

Much  of  the  processing  that  happens  in  the  brain  is  concerned  with  stages  of  perceptual  organization  for 
the  sensory  systems.  These  include  selection  processes  that  support  the  brain’s  ability  to  perceptually 
separate  and  attend  partials  of  sensory  excitations  (that  may  possess  a  high  degree  of  relevancy  to  a  task) 
from  within  a  composite  response'.  In  an  effort  to  formulate  applicable  models  for  sensory  field  responsive 
attentional  mechanisms,  augmented  Kohonen  and  Hopfield  type  organization  and  optimization  processes 
have  been  embraced  to  support  adaptive  beamforming  constructs^-^*  .  Following  the  popular  metaphor,  these 
products  are  called  "neural"  adaptive  beamformers  (NABFs).  It  is  suggested  that  these  are  generaiizable  to 
function  as  fundamental  building  blocks  in  models  of  sensory  processing,  serving  as  instantiations  of  a 
general  adaptive  beamforming  (ABF)  paradigm  that  is  useful  for  understanding  and  producing 
computational  correlates  of  cognitive  sensory  systems.  The  beamforming  paradigm  easily  integrates  the 
qualities  of  attentiveness  and  binding  when  it  is  applied  to  primitive  partials  of  sensory  excitations.  It 
emphasizes  a  neural  coding  that  is  based  upon  comparison  of  temporal  patterns  arriving  on  spatially 
separate  channels.  For  example,  beamformers  participate  in  the  transformation  of  temporal  codes  to 
spatial  codes,  i.e.,  they  produce  the  effect  that  temporal  patterns  that  arrive  on  separate  channels  are 
capable  of  activating  specific  loci  in  a  neuronal  layer  based  on  their  relative  activity. 

Ultimately,  the  fundamental  paradigms  of  the  applied  models  must  support  a  minimal  set  of  functions 
which  compose  a  comprehensive  computational  system  capable  of  autonomously  generating  percepts.  The 
literature  on  phenomenological  and  physiological  studies  of  sensory  systems  of  the  brain'suggests  that  a 
desirable  object  orientation  on  the  part  of  the  brain  is  supported  by  continual  interaction  between 
computations  occurring  within  somewhat  specialized  though  interdependent  nuclei.  This  process  often 
includes  the  integration  of  different  sensory  modalities.  However,  even  within  a  single  modality 
considerable  interactive  integration.  U  can  be  said  that  cognitive  sensory  function  in  general  encompasses 
the  simultaneous  acts  of  (1)  attending  memory  according  to  stimuli,  (2)  attending  stimuli  according  to 
iin;i. iOi>,  di.ci  (3)  attending  stimuli  and  memory  according  to  an  ongoing  "thought  process."  Thus, 
attentional  focussing  is  a  key  element  of  the  cognitive  sensory  act.  The  thought  process  is  a  director  of 
efference  or  exemplar  generation  that  expresses  an  object  hypothesis  during  performance  of  a  task  or 
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As  an  application  ot  the  general  concepts  discussed  here,  a  '■•'und  i>ioces'.<>t  i.  is'ine,  dcsel"pt‘d  tha!  i 
designed  to  be,  rn  many  respects,  a  computational  correlate  ot  animal  auditoi'.  w-aenis  1  his  '.'.oik  i- 
particularly  germane  to  sonar  system  development  and  vtiicc  •n'l iiKi  11  s.i  li  is  uiso  rolc'*  ;uH  d*  '■'iVi 

processing  and  communications  in  the  electromagnetic  rlomain  through  application  ot  its  underlsm;’, 
principles.  Similarly,  analogous  effects  are  found  in  phenomenological  siuiiies  oi'  the  human  visual  .mi 
auditory  systems,  suggesting  that  there  are  pervading  computational  [raradigms  at  vet>rk  actress  the  sen-.i  i  ', 
modalities*. 


2.  MECHANISMS 


2.1  Encoding 

What  contribution  to  the  computational  model  will  the  proper  encoding  of  the  stimulus  provide? 

Biological  systems  make  extensive  use  of  encoding  transformations.  Considering  the  auditory  system  in  thi'^ 
respect,  one  finds  the  immediate  spatial  expression  of  the  spectral  content  of  an  excitation  along  the 
basilar  membrane.  This  mapping  is  encoded  into  the  common  language  of  sensory  and  cortical  neurons  at 
the  hair  cells.  This  common  language,  in  combination  with  the  capability  of  neurons  to  learn  and 
spontaneously  generate  bursting  patterns  gives  rise  to  invariant  feature  representations,  efference. 
attentiveness,  memory,  a  retained  model  of  the  world,  and  ultimately  consciousness. 

In  our  model  for  cognitive  auditory  processing,  the  ability  to  form  an  impression  of  an  object,  i.c.,  to 
recognize  an  object  by  some  relatively  invariant  quality  of  the  sound  it  makes,  will  be  emphasized.  In 
order  to  achieve  a  frequency-shift-invariant  recognition  of  a  source  of  sound,  thereby  emphasizing  timbre 
over  pitch  for  recognition  purposes,  the  encoding  such  as  is  done  by  the  basilar  membrane  and  hair  cells  is 
essential.  The  frequency  content  of  each  partial  is  represented  by  place  in  the  tonotopic  arrangement.  For 
computational  purposes,  the  phase  is  encoded  in  the  quadrature  coding  discussed  under  2.2  Timing  and 
phasing.  Thus  it  is  only  necessary  that  the  temporal  pattern  of  the  propagating  excitation  represent  the 
amplitude  of  the  partial  as  a  function  of  time.  As  a  NABF  adaptively  weights  the  incoming  partials,  it  can 
select  a  group  of  responses  out  of  the  tonotopic  arrangement  that  exhibit  the  correct  relative  patterns  of 
amplitude  versus  time  as  being  representative  of  a  particular  timbre  or  quality  of  the  sound  that  is 
recognizable  independent  of  pitch.  Thus  the  recognition  of  a  musical  instrument  can  be  achieved 
independent  of  what  note  is  being  played;  the  train  whistle  can  be  recognized  independently,  to  a 
significant  degree,  of  the  current  Doppler  effect  (or  by  using  the  Doppler  effect  to  advantage  because  the 
frequency  modulation  itself  will  be  a  recognizable  quality’). 

This  encoding  seems  to  be  achieved  in  the  auditory  system  through  the  physiology  of  the  hair  cells,  their 
stereocilia  and  links.  The  rectified  charge  flow  rate  representation  of  amplitude  is  summated  (integrated) 
and  the  rate  of  ceil  potentiation  is  reflected  in  the  rate  of  firing  of  the  cell.  The, relationship  between  tiie 
firing  rate  and  the  stimulus  amplitude  may  have  statistical  properties  dependent  upon  stimulus  noise  or 
internal  neural  noise  as  in  the  stochastic  resonance  phenomenon*. 

2.2  Timing  and  Phasing 

The  fundamental  ability  to  manipulate  timing  is  important  in  encoding  and  selection  schemes.  Within 
neuronal  tissu",  an  innate  'ibility  to  support  pr<^p'’gaf<nn  speeds  and  latency  periods  over  w'ide  faugc;  n.xi;:: 
as  part  of  the  chemistry  of  the  physical  mechanisms.  To  enhance  the  modeling  of  the  timing  action  of  the 
computational  network,  both  in-phase  and  time-lagged  or  phase-shifted  versions  of  the  inputs  are 
presented.  This  gives  the  network  the  ability  to  operate  in  a  quasi-analytic  domain. 
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accomplished  by  introduction  i.)f  firupagation  time  lag  anil  neui--nai  (''  i-  n  e  lateiu  s  li  the  o  io,;  -  ■  : . 
lag  in  the  processing  is  reversed  from  that  produced  by  the  ■'patral  arrangement  of  tire  sensut  >  t  o  i  ,  .  .  : 
angle  of  incidence,  then  the  excrtalions  will  be  added  m-phase  lesultmg  in  a  maxiniuin  i  '.[r  ui  f  ;  :a at 
angle  of  incidence.  Simple  time  lag  is  very  effective  in  spatial  processing  for  broadbami  signaii  an  ! 
produces  appropriate  phasing  across  all  frequency  components  of  a  stiniuius  simuhaneously  '■  Ai  n  ti-.e 
stimulus  is  an  aperiodic  waveform  then  there  is  no  response  ambiguits  versus  angle,  r.e  .  there  i^  oi:i.  ...n 
processing  time  lag  that  produces  tlie  maximum  correlation. 

Concerning  spatial  mapping,  and  as  a  matter  of  computing  convenience,  the  question  arises  wiu  tiu  r 
under  certain  circumstances  the  timing  quality  of  neural  interaction  may  be  modeled  by  phasing,  i  e  .  1  v  .i 
quadrature  hybrid  technique  for  each  frequency  band  or  aud  '.ory  fibre.  It  seems  that  in  the  ca-e  of  tlie 
auditory  system  in  humans,  the  distance  between  the  ears  is  approximately  half  a  wavelcnglh  in  air  l\>r 
1000  Hertz.  Therefore,  the  maximum  internal  delay  that  would  be  necessary  to  compensate  for  piopaganun 
time  differences  would  be  less  than  half  a  period  of  the  1000  Hertz  wave  in  air  as  long  as  spatial  masipitii' 
is  restricted  to  fibers  responding  at  frequencies  below  1000  Hertz.  I  hus,  in  the  computational  model  at 
least,  the  analytic  computation  of  phase  can  be  used  to  represent  timing  for  a  subset  of  the  audnoiv  fibre 
Another  question  that  arises  is;  does  biological  tissue  use  analytic  encoding? 

To  control  phase  by  computing  in  the  analytic  domain,  quadrature  shifted  excitations  are  included  m  liie 
input  suite  on  separate  channels.  To  compute  the  quadrature  shifted  version  of  a  finite  bandwidth 
excitation,  the  Hilbert  transform,  x{t),  of  an  excitation,  x(f),  may  be  approximated  numerically  as  an  FIR 
Hilbert  transformer  filter  with  frequency  response 


H{Q)  = 


-j,  0  <  Q  <  7t  I 

J,  -  7t  <  Q  <  0  j 


Ci 


where  is  the  dimensionless  frequency’.  The  appropriate  set  of  weights  is  conceivably  Jearnable  by  a 
neuronal  processing  element.  Furthermore,  if  the  frequency  partitions  are  narrow,  the  derivative,  x{t). 

divided  by  the  center  frequency  of  the  band,  0)c,  is  a  good  enough  approximation  to  the  quadrature  input 
in  some  cases,  i.e., 


xU)  ~  x{t)/(xJc  =  yZ(o),7a)c)c{co,)e'“''  =  jx{t)  o ' 

I 

where  the  c(0))  are  the  complex  fourier  coefficients  for  the  series  approximation. 

In  well-known  treatises  on  adaptive  processing  it  has  been  shown  that  the  addition  of  a  second  lag  (a 
third  tap)  of  the  input  can  increase  the  adaptive  beamformers  ability  to  enhance  the  signal-to-noise  ratio 
of  a  broadband  signal  in  noise  and  interference'”.  The  third  tap  is  an  inhibitory  synapse  relative  to  the 
first  tap,  since  it  produces  the  negative  of  the  first  tap  input  at  the  center  frequency  of  the  band. 

2.3  Hopfield-type  Optimization  for  Selective  Spatial  Focussing 

An  optimization  scheme  has  been  developed  that  has  the  capability  to  adaptively  adjust  timing/phasing  jo 
do  spatial  beamforming.  An  example  architecture  is  illustrated  in  Fig  '  The  Hopfield  crorebar  ci’zun 
arrangement  '  is  used  as  a  computational  kernel.  The  output  voltages  of  the  circuit  are  to  represent  the 
weights  on  an  adaptive  combiner'^  In  order  to  formulate  the  beamformer  mechanisro  so  that  it  responds 
adaptively  to  inputs,  the  minimum  mean  square  error  problem  is  posed  and  the  currents  and  connectivities 
are  solved  for  as  functions  of  the  inputs.  The  resulting  exprc-ssions  are 
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where  X  is  a  response  latency  period,  and  i',(/  — t)  is  the  output  amplitude  at  the  end  of  the''previous  epoch 
from  the  ith  element.  In  the  case  of  discrete  time-step  simulations,  the  expectation  value  is  usually 
evaluated  by  summing  . 

The  performance  of  the  Crossbar  Adaptive  Beamformer  (CABF)  was  validated  against  composite  sounds 
of  a  real  sonar  scene  impinging  upon  a  spatially  complex  array.  The  data  were  obtained  from  the  Sonar 
Thinned  Random  Array  Program  (STRAP).  Fig.  2  depicts  the  spatial  arrangement  of  1 1  sonobuoys  that 
were  dropped  in  the  Atlantic  ocean.  A  known  source  was  active  at  a  distance  of  approximately  10  miles. 

It  consisted  of  two  frecjuencies,  seven  and  eleven  Hertz.  Figs.  3  and  4  show  spectral  densities  from  various 
channels.  Notice  the  inconsistency  across  the  channels. 

The  temporal  recordings  made  at  these  buoys  were  played  into  the  beamformer.  Fig.  5  shows  the 
adapted  sensitivity  of  the  CABF  as  a  function  of  time.  The  CABF  is  correctly  attending  the  desired  signal 
at  approximately  41  degrees.  These  results  are  very  good  when  you  consider  that  no  spectral  preprocessing 
was  performed,  i.e.,  the  desired  signal  was  still  mixed  with  the  other  interfering  components  at  a  level  of 
approximately  -20dB  with  respect  to  some  higher  frequency  components  (Fig.  3).  More  tests  are  being 
performed  in  scenarios  wherein  interfering  signals  are  arriving  concurrently  with  the  signal  of  interest  and 
from  a  variety  of  directions  (see  Mutual  Inhibition  below). 

2.4  Adaptive  Temporal  Sifting 

It  has  been  demonstrated  by  many  studies  and  by  our  own  experience  listening  to  monaural  radio  sets 
that  the  human  auditory  system  need  not  have  spatial  cues  in  order  to  sort  out  sounds  that  are 
simultaneously  incident  on  our  ears.  To  give  this  sorting  capability  to  a  processor,  a  temporal  sifting 
procedure  may  be  formulated  using  an  augmented  form  of  the  Kohonen  self-organization  procedure*’.  The 
modified  learning  rule  allows  temporary  storage  of  multi-dimensional  n-vectors*'  that  represent  the  average 
vectors  of  statistically  meaningful  groupings  or  classifications  of  the  input  vectors.  The  action  of  the 
so-called  Multi-vector  Adaptive  Beamformer  (MABF)  is  pictorially  represented  in  Fig.  6.  Temporal  sifting 
and  short-term  memory  formation  is  accomplished  according  to 

W"^  =  W^'‘‘  +  a{x,-Ci,)Ad^  (6) 

where  Wi  is  the  "weight  plane",  expressed  as  a  bi-vector  in  this  case,  of  th^ith  processing  element  (PE)  of 
a  layer  of  PEs  that  receives  a  fan-out  of  the  inputs,  a  is  the  learning  rate,  Cj*  is  the  projection  of  the  kth 

input  vector,  x*,  onto  Wt,  normalized  to  unit  length.  dik  =  Xf  (the  bar  denotes. normalization),  and 
the  symbol  A  denotes  the  wedge  product.  For  the  case  of  temporal  learning,  the  input  vectors  are  formed 
from  a  tapped  delay  line. 

2.5  Quasi-recursive  and  Hcbbian  characteristics 

The  functional  qualities  of  continuity,  accommodation,  and  memory  are  supported  by  quasi-recursive 
(QR)  and  Hebbian  arrangements  whereby  the  adaptive  mechanisms  receive  feedback  from  the  output  of  the 
total  adaptive  process.  Both  types  are  depicted  in  the  system  building  blocks  in  figs.  7  and  8.  They  are 
distinguishable  by  the  connections  to  the  adaptive  mechanism. 
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1  iu“  (,)R  nu'Jlian isiii  priu  Kit.'s  hohavior  SDinevHliat  anal' :  i  Iho  !!i[i  tl'.'P  ilia'  is  used  ni  (iieiial  li.'i;:'' 
-State  machines.  In  this  case  luiweser  there  are  an  nitmi'e  luinihei  ut'  p./'silile  states,  depeiuient  upuii  tiii- 
exemplar  that  is  being  presented  and  the  cuntent  oi  the  •'imuila-  t  leld.  1  hus,  d'  a  cjuasi -  recursive  NAISl 
(QR-NABF)  finds  vvhat  it  is  looking  for  in  the  seti-soi'.  t’vcitaiion.  it  locks  on  to  it,  forming  a  ipiiie  stafie 
state  that  persists  even  if  the  desired  excitation  is  tent(iorar ilv  mteifeied  with  Thus,  Uie  QR  -NAUI  f /in; 
a  short-/medium-term  memory  and,  what  is  more,  comcideniaih  aitetuls  ami  verifies  the  occurrence  i..t  iln 
desired  excitation.  An  architecture  for  the  implementation  of  (jR  NABl  s  is  discussed  below'  1  lie 

equat'cns  (1)  and  (5)  apply  with  each  occurrence  of  t,  replaced  by  p.,  where  the  multiple  outputs,  f,  .ne 
each  phase-centered  on  the  ith  sensor; 

connectivity  T,,it)  =  \[  ,  I'.itih  -  0  i  i 

ion  flow  I,U)  =  +  T  -fW'l  -  jy,{t~T)\[^x\(n)yAr\)(ln  ,8, 

[he  Hebbian  arrangement  correlates  the  feedback  from  the  output  with  the  corresponding  inputs  to 
determine  the  synaptic  modification.  Equations  (4)  and  (5)  apply  with  changes  to  reflect  the  Helib-like 
learning  rule; 


connectivity  T.jit)  =  -HU  •  ’^>>=0 


ion  flow  /,(/)  =  Y  ^  lU 


2.6  Mutual  inhibition 

In  order  to  utilize  resources  efficiently  and  have  the  capability  of  perceptually  separating  mixtures  of 
stimuli  the  neural  elements  must  interact  in  a  way  that  forms  an  organizational  network.  A  sensory 
hierarchy  is  one  such  organization  (as  it  turns  out,  the  most  easily  implemented).  Building  blocks  for  a 
hierarchical  assembly  are  represented  diagrammatically  in  Fig.  7  and  Fig.  8.  The  "beam  group"  symbols 
each  represent  a  set  of  beams  that  are  derived  by  displacing  the  phase  reference  to  particular  sensors  or 
channels.  Thus,  a  beam  group  has  an  output  channel  for  each  input  channel.  The  outputs  of  each  module 
goes  to  the  inputs  of  a  similar  module,  and  so  on.  In  this  kind  of  arrangement,  elements  that  are 
responding  to  a  particular  stimulus  inhibit  other  elements  lower  in  the  hierarchy  from  responding  to  it  and 
thereby  free  them  to  respond  to  other  stimuli  that  may  be  concurrently  active. 

A  neurobiological  correlate  to  this  action  may  be  suggested  by  recent  work  regarding  the 
inferior-temporal  (IT)  cortex  of  the  rhesus  monkey'^  It  was  found  that  some  cells  in  that  region  have  a 
reduced  response  over  time  to  repeated  stimuli  while  maintaining  substantial  response  to  new  stimuli.  It 
could  be  hypothesized  that  other  cells  are  selecting  the  repealed  stimuli  and  inhibiting  their  propagation  to 
the  cells  in  IT,  thereby  forming  a  novelty  filter  effect. 

A  hierarchical  beamforming  approach  has  been  used  for  similar  purposes.  The  inhibitory  effect  comes 
about  through  the  formation  of  notch  beams  that  pass  the  excitation  to  lower  levels  in  the  hierarchy.  First, 
a  quasi-recursive  beamformer  enhances  its  sensitivity  to  a  particular  excitation  as  described  previously. 
Next,  a  combination  of  inhibitory  and  excitatory  connections  to  another  element  produces  the  notch  beam, 
i.e.,  it  produces  a  minimization  of  response  in  the  dimensional  ranges  where  the  initial  elements  response  is 
maximized. 

In  Figs.  9  and  10,  parts  a  and  b  show  the  NABF  receptivity  as  a  function  of  the  angle  of  incidence  at 
the  first  (highest)  and  second  levels  of  the  hierarchy,  respectively.  Two  identical  signals  are  incident  at 
approximately  35  and  -30  degrees.  In  Fig.  9,  the  signal  onsets  are  simultaneous  and  the  NABF  pair  are 
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Contused  In  l  igs  10  niul  I  1.  one  signal  onset  is  delayed  approMin.itels  10  iiiilii',ix;..iui  Iomu  ;jie  e,:,  ! 
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contusion  evident  in  1  ig.  9  is  eliminated.  Recursion  is  applied  in  I  ig  11  but  nut  m  tie.s  ')  oi  10 

2.7  .Adaption  of  receptive  field  within  internal  representations 

If  neurons  can  adjust  their  receptive  fields  as  the  ABF  paradigui  suggests  (by  adaptively  v.'eightiiig 
synapses)  then  there  may  be  attentional  shifting  within  the  internal  physical  representations  generated  b\ 
sensory  systems.  A  phenomenon  that  has  been  observed  recently  in  visual  processing  in  the  parietal  coiiex 
of  monkeys'*’  suggests  that  neurons  have  the  ability  to  coordinate  representational  shifts  w  ith  movements  ol 
the  eye.  In  the  context  of  the  auditory  system,  a  related  capability  may  account  for  pitch- independent 
recognition  of  a  musical  instrument.  Thus,  in  a  cognitive  sensory  system  model,  the  ABF  paradigm  will  bf 
utilized  to  realize  active  recognition  memory  that  can  move  the  receptive  fields  of  its  elements  along 
internal  representations.  With  regard  to  hearing,  the  action  of  the  cochlear  partition  contributes  to  an 
internal  representation  that  preserves  patterns  of  excitation  due  to  the  logarithmic  best-frequency 
dependence  as  a  function  of  distance  along  the  basilar  membrane. 

3.  COGNITIVE  SENSORY  SYSTEM 

The  system  architecture  is  very  important,  not  only  because  it  provides  for  the  funnelling  of  outputs  of 
one  process  into  the  inputs  of  another,  but  also  because  the  architecture  supports  multidimensional 
encoding  transformations,  e.g.,  topological  mappings  that  have  important  relationships  to  external  space,  or 
that  allow’  reduction  of  the  dimensionality  of  information  to  be  axonally  propagated  In  addition,  the 
somewhat  specialized  sub-processes  of  the  system  are  interdependent  and  require  robust  interconnection. 
Therefore,  the  neural  computation  approach  is  ideal  and,  along  with  the  considerations  reviewed  in  the 
INTRODUCTION  and  MECHANISM  sections,  leads  to  a  plausible  processing  scheme  for  selectively 
attending  partials  of  complex  sensory  excitations. 

There  is  an  issue  of  fundamental  importance  raised  by  a  cortical  beamforming/recognition  approach  with 
regard  to  generation  of  exemplars  used  for  attentionally  directed  segmentation;  the  issue  is  whether  or  not 
an  exemplar  is  generated  in  cortex  and  passed  to  some  more  peripheral  part  to  the  sensory  system  (perhaps 
an  intermediate  stage)  via  efferents.  This  would  facilitate  the  attentional  function  at  early  stages  as  has 
been  observed  and  it  would  also  facilitate  internally  generated  "visualization"  about  sensory  experience,  i.e  , 
it  would  facilitate  the  ability  to  visualize  some  sensory  happening.  This  would  happen  without  an 
immediate  incoming  sensory  prompt  and  would  be  generated  out  of  associations  made  during  thought 
causing  the  enlistment  of  sensor  areas  for  visualization  by  the  production  of  efference  to  intermediate 
and/or  peripheral  sensory  areas.  In  either  case,  the  efference  is  produced  by  associations  made  in  cortex  .  . 
by  association  between  memory  and  afferent  sensory  activity  in  one  case  and  between  memory  and  thought 
in  the  other. 

In  the  attentional  effect  produced  by  the  beamformer,  the  efference  serves  as  an  exemplar.  Thus,  a 
resonance  can  be  achieved  through  interaction  of  sensory  activation,  associational  memory,  and  thought 
(symbolic  processing  as  can  be  generated  by  expert  systems)  to  achieve  focussing  on  portions  of  the  sensory 
activity  remembered  or  thought  (hypothesized)  to  be  mission-relevant  (or  survival-relevant). 

In  experiments  related  to  this  issue,  Metzner  at  Scripps  Institute  of  Oceanography,  University  of 
California,  reported  an  efference  within  neural  tissue  whereby  transmissions  of  the  horseshoe  bat  are 
compared  with  echoes  in  order  to  sense  the  Doppler  and  adjust  its  emission  frequency  accordingly.  This  is 
in  contrast  to  the  idea  that  the  bat  listens  to  its  own  emission  in  order  to  generate  the  exemplar.  This 
brings  up  the  question  whether  the  cortical  oscillations  being  reported  in  the  literature  are  active  generators 
of  "efference." 
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A  conceptiiai  onoi'.  leu  anti  some  key  elements  of  a  comprehensive  processing  sclreme  are 
depicted  in  figures  12  and  la,  respectis eh'.  The  conceptual  overview  is  a  simplified  rermesentatioti  tltat 
includes  four  principal  functions:  (1)  the  partitioning  function,  (2)  the  selection  function.  (.1)  position  and 
motion  determination,  and  (4)  recognition.  In  reality,  these  functions  are  not  performed  separately  lhe> 
are  provided  by  the  interacting  elements  of  the  processing  scheme. 

The  key  elements  of  the  processing  scheme  are:  (1)  multiple  band  filtering,  (2)  binaural 
correlation,  (3)  spatial  adaptive  beamforniing,  and  (4)  temporal  adaptive  beamforming,  [n  practice, 
multiple-band  filtering  corresponding  to  the  cochlear  filtering  indicated  in  figure  13  is  performed  using  a 
scaled- wavelet  formulation,  providing  a  spread  of  bandwidths  associated  with  the  various  bcst-frequencit'^ 
It  is  recognized  that  this  model  cannot  account  for  the  sharpness  of  the  cochlear  response  function  near  the 
best  frequency"'.  Spatial  mappings  have  been  observed  in  the  colliculus  in  some  vertebrates'''  and  in 
cortical  field  A I  of  the  cat  . 

The  processor  contains  a  band- wise  spatial  mapping  that  receives  excitations  from  the  partitioned  output 
of  the  cochlear  process,  i.e.,  the  array  of  bandpass  filters.  The  spatial  mapping  is  supported  by  the 
confluence  of  afferent  excitation  from  the  sensors  at  intermediate  computational  nuclei  (labelled  SOC  in 
Fig.  13).  An  adaptive  mechanism  supplies  attentional  focussing  while  the  excitations  are  spaiiallv  mapped, 
creating  areas  of  enhanced  activity.  This  function  may  be  a  correlate  of  the  activity  of  the  dorsal  cochlear 
nucleus  (DCN).  Intermediate  between  the  cochlear  processing  and  the  band-wise  spatial  mapping  is  an 
adaptive  spatial  process  (not  depicted)  provided  by  the  NABF.  The  darkened  areas  represent  those 
attended  (emphasized)  by  the  NABT.  Thus  the  spatial  layer  acts  as  a  sieve,  passing  attended  stimulus 
partials. 

The  spatial  mappings  can  be  related  to  the  beamformer  sensitivity  maps  of  figures  9  through  !  1  w’here  a 
single  row  of  the  spatial  map  as  a  function  of  time  is  plotted  contiguously  down  the  page.  These  maps 
may  be  thought  to  represent  the  activity  of  a  layer  of  beamformers  with  relatively  fixed  directional 
preferences.  What  is  the  purpose  of  forming  a  topological  organization  of  the  beamformers?  The 
topological  mapping  creates  an  organization  by  which  the  cells  for  the  various  auditory  bands  which 
respond  to  a  given  object  in  space  are  close  together.  This  simplifies  the  projection  of  the  output  of  the 
spatial  neurons  to  the  cortical  area  where  recognition  is  accomplished.  This  organization  is  also  beneficial 
in  the  digital  signal  processing  application,  though  the  units  are  not  actually  arranged  spatially  but  are 
arranged  by  ordinal  number  instead. 

Notice  that  a  loop  has  been  formed,  because  the  output  of  the  spatial  map  projects  as  input  to  the 
recognition  area,  the  output  of  which  was  utilized  in  the  formation  of  the  spatial  map.  In  the  biological 
case,  it  is  not  clear  whether  this  system  constantly  feeds  back  on  itself  or  if  there  is  an  afferent  wave  of 
activity  followed  by  an  efferent  wave  or  vice  versa.  In  the  case  of  the  computational  model,  it  can  be 
done  either  way  and  perhaps  an  investigation  of  (his  will  lead  to  some  conclusions.  It  could  be  that  the 
loop  leads  to  oscillations  in  some  circumstances.  If  it  does,  the  relationship  of  the  oscillations  nia\  be 
studied  in  light  of  recent  observations^'. 

In  the  computational  model,  the  projections  from  the  spatial  map  are  Input  to  a  M.ABF-  process.  The 
overall  action  of  the  recognition  MABF  is  to  segment  and  identify  patterns  of  temporal  activity  across  the 
auditory  bands  which  are  established  in  the  c'lchlea.  Each  spatial  stimulus  segment  is  again  segmented 
temporally  according  to  memory  by  creating  a  time  dependent  sensitivity.  The  MABF  attempts  to  create  a 
stimulus  partial  which  matches  the  temporal  characteristics  of  each  band,  the  total  effect  being  to  match 
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the  spectral  content  as  a  function  of  time,  inciiuliii!’  lelaine  [itiase  vaiiaticii.,  •  ;  o;  :  mr;:;  ; 
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responses  across  the  bands,  i.e.,  the  temporal  qualities  of  all  the  individual  iiequt  n,.  .  i  i:;.i  r,  fei;. s 
assessed  simultaneously.  I  rcquency  modulation  in  the  stimulus  will  a()peai  a^  i eciiipii/al  leioi'  i a! 
variation  in  the  bands.  In  die  training  mode,  memories  are  establishecf  as  a  set  d  'Aei.i’h'ini  , 

The  main  function  of  the  recognizer  is  to  attend  memory  according  to  the  stimulus  ()(ii>  the  temp  uaii, 
varying  activities  of  the  attended  spatial  segments  elicit  memories,  because  ilies  ate  ii. mger  Imtialls. 
however,  the  system  may  not  be  attentionally  focussed  and  the  performance  of  the  .'s.AHf  on  comp<)  ate 
waveforms  becomes  important.  In  some  cases,  individual  sounds  may  not  he  discerned  .vithoui  intei'.en!;  : 
of  a  thought  process. 

When  a  memory  is  elicited,  a  partial  of  the  stimulus  is  produced  through  the  action  of  the  lempoial 
beamformer,  i.e.,  when  a  beamformer  wins  then  the  temporal  vector  associated  with  that  memory  is 
considered  a  partial  of  the  stimulus  (a  significant  one).  This  partial  is  fed  back  to  the  spatial  mapping 
process,  resulting  in  attention  to  or  a  focussing  upon  the  spatial  sector  from  where  the  partial  came 


4.  CONCr.USIONS 

Adaptive  "beamforming"  can  play  multiple  roles  in  comprehensive  sensory  processing  systems  and  serves 
as  a  paradigm  that  applies  as  well  to  patterns  as  are  observed  in  neuronal  responses  in  the  cochlear  nucleus 
and  superior  colliculus  as  it  does  to  simple  spatial  and  temporal  filtering  patterns.  Some  neural 
arrangements  for  adaptive  beamforming  outlined  here  have  been  dentonstrated  to  function  as  expected  on 
sea-test  data  and  in  the  laboratory.  Building  blocks  for  groups  of  mutually  inhibitory  beamformers  were 
also  outlined,  and  the  operation  of  a  pair  of  them  was  demonstrated.  They  were  shown  to  respond 
correctly  under  the  conditions  of  simultaneous  incidence  of  multiple  identical  (or  very  similar)  stimuli, 
wherein  a  single  NABF  would  be  confused. 
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Fig.  1.  An  adaptive  beamforming  architecture  that  uses  the  Hopfield 
arrangement  for  solving  the  optimization  problem.  The  optional  "third  tap,” 
which  is  an  inhibitory  connection  relative  to  the  first  tap,  is  represented  in 
dashed  lines. 


Fig.  2.  The  deployment  pattern  for  the  1 1  buoys  relevant  to  the 
test  data  from  the  Sonar  Thinned  Random  Array  Program  (STRAP). 
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Fig.  3.  Spectral  density  plots  for  the  data  from  the  channel  1  buoy, 
varying  frequency  scale.  The  signal  of  interest  is  a 
combination  of  7  and  II  Hertz. 
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Fig.  A.  Spectral  density  plots  for  three  different  buoys; 

fixed  frequency  scale.  The  region  of  interest  is 
emphasized,  ^^ote  variation  from  buoy  to  buoy. 
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Fig.  5.  The  automatically  adapted  beam  sensitivity; 
result  with  no  pre-filtering. 


the  beamforming 
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Fig.  6.  Illustration  of  constructs  relevant  to  a  bi-vector  instantiation 
of  the  multi-vector  beamforming  paradigm. 


Fig.  7.  Illustration  of  the  quasi-recursive  (QR)  computational 
module  for  hierarchical  processing. 
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Fig.  8.  Hebbian  module  for  hierarchical  interaction. 
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Fig.  9.  First  (a)  and  second  (b)  level  hierarchical  beamformers. 

They  are  confused  when  two  identical  stimuli  are  incident 
from  35  and  -30  degrees  and  with  simultaneous  onset. 
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Fig.  12.  Conceptual  overvie  of  the  sensory  processing 
architecture. 
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Fig.  13.  An  architecture  for  cognitive  sensory 
processing  with  auditory  system  analogues. 
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